Establishment and use of time mapping based on interpolation using low-rate fingerprinting, to help facilitate frame-accurate content revision

ABSTRACT

A media client ascertains a plurality of matching points between (i) query fingerprints representing a media stream being received by the client and (ii) reference fingerprints, each identified matching point defining a respective match between a query fingerprint that is timestamped with client time defined according to a clock of the client and a reference fingerprint that is timestamped with true time defined according to a timeline within a known media stream. Further, the client performs linear regression based on the timestamps of the ascertained plurality of matching points, to establish a mapping between true time and client time. The client then uses the established mapping as a basis to determine a client-time point at which the client should perform an action with respect to media stream being received by the client. And the client performs the action at the determined client-time point.

REFERENCE TO RELATED APPLICATION

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/700,660, filed Jul. 19, 2018, the entirety of which is herebyincorporated by reference.

This application is also related to U.S. patent application Ser. No.16/166,971, filed Oct. 22, 2018, the entirety of which is herebyincorporated by reference.

BACKGROUND

A typical media client operates to receive an analog or digital mediastream representing media content such as video and/or audio content andto output the media content and/or forward the stream for presentationof the content on a user interface such as a display screen and/or anaudio speaker. Examples of such clients include televisions, computermonitors, projection systems, loudspeakers, headphones, set top boxes(e.g. cable or satellite TV receivers), digital video recorders, radios,personal computers, mobile communication devices, gaming consoles,streaming media players, and the like.

By way of example, the media client could be a television, which couldreceive a media stream (e.g., an over the air broadcast stream, a mediastream output from a set top box, a media stream played from a digitalstorage medium, an Internet stream, or another such stream) and couldpresent the media content of that stream to a user. As another example,the media client could be a set top box, which could receive a broadcaststream from a multi-channel video program distributor and could outputthe media content of that stream via a High-Definition MultimediaInterface (HDMI) cable or other interface to a television and/oraudio/video receiver for playout. As still another example, the mediaclient could be a digital video or audio recorder, which could likewisereceive a broadcast stream but could store the stream for later playout.And as yet another example the media client could be a loudspeaker orheadphones, which could receive an audio stream from a radio, computer,or other receiver or audio storage device and could present the audiocontent of that stream to a user. Numerous other examples are possibleas well.

SUMMARY

When a media client is receiving a media stream, it may be useful incertain situations for the client to revise a portion of the stream'smedia content. For example, it may be useful for the client todynamically replace an ad or other segment of the media content with areplacement ad or with other replacement content, so that when thecontent is played out to a user, the user would receive the replacementad in place of the originally included ad. As another example, it may beuseful for the client to dynamically supplement a portion of the mediacontent with overlay or split-screen content, such as channelidentification, context information, ad content, or the like, so thatwhen the content is being played out to a user, the user would receivethe supplemental content together with the originally included content.

Further, it may be desirable for the client to perform such contentrevision at a specific time point within the media stream. For instance,if the client is to dynamically replace an existing ad with areplacement ad, it may be desirable for the client to position thereplacement ad at a time in the media stream when the existing ad wouldhave started. Likewise, if the client is to supplement a specificportion of the media content with an overlay related to that portion, itmay be desirable for the client to start the overlay when that portionstarts. Other examples are possible as well.

More particularly, the media stream at issue could define a sequence offrames, such as video frames and/or audio frames, and it may bedesirable for the client to perform the content revision on aframe-accurate basis. For instance, for ad replacement, it may bedesirable for the client to insert the replacement ad in place of theexisting ad starting precisely at the first frame of the existing ad.And for adding an overlay to a specific portion of the media stream, itmay be desirable for the client to start the overlay at or in specificrelation to the first frame of that portion of the media stream.

To facilitate having the client perform this content revision, theclient may need to learn the specific time point where the revision isto occur. In practice, however, the revision time point may be definedon a timeline that is unknown to the client. For instance, the revisiontime point may be defined on a reference timeline within the mediastream itself, but the client may not have context for that timeline.Therefore, providing the client with that revision time point may notenable the client to perform the revision at the desired time point inthe media stream.

As an non-limiting example of this, consider a media stream that definesan ad pod that could appear within a television broadcast. An ad podcomprises a sequence of ads, such as television commercials. With thisexample, the client might be a television that is receiving andpresenting a linear broadcast feed of a particular television channel,and the ad pod might appear at any moment within that feed. Especiallywith a live broadcast such as a sporting event, for instance, the timewhen the ad pod appears in such a feed may be unpredictable.Alternatively, the ad pod could appear at a specifically scheduled time.

In this example, the desired revision could be replacement of aparticular ad within the ad pod (e.g. substituting for the ad adifferent ad or other replacement content). For instance, if the ad podis a sequence of ads A1, A2, A3, A4, the desired revision could bereplacement of ad A3.

In this situation, the ad pod may define an internal timeline, startingat time T=0 and continuing to the end of the ad pod, and ad A3 may beginat time T_(A3) on that timeline. However, the client may lack contextfor that timeline. The client may have its own internal clock that has atimeline (e.g., time of day or other internal time reference) defining“client time.” But without advanced knowledge of where the ad-podtimeline exists within client time, merely given the value T_(A3) as thedesired content-revision time point would not enable the client toperform the ad replacement when desired.

One way to address this issue is to perform fingerprint matching,comparing timestamped query fingerprints of the media stream beingreceived by the client with timestamped reference fingerprints of theknown media stream such as the known ad pod sequence.

For instance, the client could generate query fingerprints of frames ofthe media stream that the client is receiving and could timestamp thosequery fingerprints based on the client's internal clock, and a computingsystem could receive those timestamped query fingerprints for analysis.Further, the computing system could be provisioned with referencefingerprints of frames of the known media stream such as the known adpod sequence, timestamped based on reference time or “true time” definedalong a timeline within the known media stream.

The computing system could thus compare the timestamped queryfingerprints with the timestamped reference fingerprints in an effort tofind a matching point where a query fingerprint matches a referencefingerprint, and upon finding the matching point, could conclude thatthe timestamp of the matching query fingerprint corresponds with thetimestamp of the matching reference fingerprint.

Establishing this correlation between the timestamps of the matchingquery and reference fingerprints could constitute establishing“synchronous lock” or time mapping between client time and true time.For instance, a difference between the timestamp of the matching queryfingerprint and the matching reference fingerprint could define a timeoffset between true time within the known media stream and client timeas the client receives the media stream. Given knowledge of a true-timepoint of a frame within the known media stream, the computing systemcould thus use the established synchronous lock to determine theassociated client-time point of the same frame in the media stream beingreceived by the client. For instance, the computing system could applythe time offset between true time and client time, in order to determinethe client-time point corresponding with the given true-time point.

Continuing with the ad-pod example above, the computing system could usethe established synchronous lock to map the true-time value T_(A3) to aclient-time value denoting the client-time point when the client shouldreplace ad A3 in the media stream being received by the client (e.g. theclient-time point where the client should start the ad replacement). Andthe client could then perform the replacement of ad A3 at thatdetermined client-time point.

A technological problem that can occur with this process, however, isthat the client may not generate query fingerprints fast enough tofacilitate high-confidence fingerprint matching between the queryfingerprints and the reference fingerprints. In particular, to helpconserve limited processing power, memory, and/or other resources at theclient, the client may be arranged to generate fingerprints at arelatively low frame rate, which may be far below the frame rate of themedia stream that the client is receiving. As a result, the detectedfingerprint matching may be inaccurate and the established synchronouslock may be inaccurate.

For example, with video having 30 frames per second, the client might beset to generate query fingerprints at a low frame rate such as 2 frameper second. But performing fingerprint matching at that low rate mayresult in a low-confidence fingerprint matching.

One reason for this difficulty is that the media stream may includevarious sub-sequences of identical or nearly identical frames,corresponding with portions of the media sequence where the mediacontent does not change materially over a period of time. An example ofthis could be video that presents a still image for a period of time. Inthat scenario, a query fingerprint might match multiple consecutivereference fingerprints within a defined matching tolerance.

As a result, the computing system's detecting of a given matching pointmight be relatively low-confidence. And consequently, the computingsystem's established mapping between true-time and client-time would bepotentially inaccurate, which could give rise to user-experienceproblems. For instance, inaccurate time mapping could result in theclient performing a directed content revision too early or too late.With content replacement (e.g., ad replacement), a user could perceivethis poorly timed content revision as a discontinuity or inconsistencyin the presented media content.

Per the present disclosure, this issue can be at least partly resolvedby taking into account multiple detected matching points andinterpolating between those points to establish an improved mappingbetween true time and client time.

In a representative implementation, given low-rate query fingerprintsgenerated by the client, the computing system could conduct fingerprintcomparisons as discussed above, and the computing system could detectmultiple matching points over time, with each matching pointrepresenting a determined correlation between a true-time point and aclient-time point. As the computing system detects these matchingpoints, the computing system could then programmatically establish ascatter plot of the points and perform linear regression to establish abest-fit line that represents an improved mapping between true time andclient time.

The computing system could then use this improved mapping as a basis topredict what point in client time will correlate with an upcoming pointin true time, so as to facilitate having the client perform a contentrevision or take other associated action (e.g., preparation for contentrevision) at the upcoming time point. For instance, the computing systemcould use the improved mapping as a basis to predict what point inclient time will correlate with a true-time reference timestamp of anupcoming frame where the client is to perform a content revision. Andthe computing system could then cause the client to perform the contentrevision at that predicted client-time point.

These as well as other aspects, advantages, and alternatives will becomeapparent to those of ordinary skill in the art by reading the followingdetailed description, with reference where appropriate to theaccompanying drawings. Further, it should be understood that thedescriptions provided in this summary and below are intended toillustrate the invention by way of example only and not by way oflimitation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified block diagram of system in which disclosedfeatures can be implemented.

FIG. 2 is an example scatter plot of matching points each having arespective true-time value and a respective client-time value.

FIG. 3 is a flow chart depicting operations that could be carried out inaccordance with the disclosure.

FIG. 4 is another flow chart depicting operations that could be carriedout in accordance with the disclosure.

FIG. 5 is a simplified block diagram depicting components of a computingsystem operable in accordance with the disclosure.

DETAILED DESCRIPTION

Referring to the drawings, FIG. 1 is a simplified block diagram of anexample system in which various disclosed principles can be applied. Itwill be understood, however, that this and other arrangements andprocesses described herein can take various other forms. For instance,elements and operations can be re-ordered, distributed, replicated,combined, omitted, added, or otherwise modified. Further, it will beunderstood that operations described herein as being carried out by oneor more entities could be implemented by and/or on behalf of thoseentities, through hardware, firmware, and/or software, such as by one ormore processing units executing program instructions or the like.

As shown in FIG. 1, the example system includes a media client 12configured to receive a media stream in real-time from a media source14.

In practice, the media client 12 could be configured to operate asdescribed above to receive from the media source 14 an analog or digitalmedia stream representing media content such as video and/or audiocontent and to output the media content and/or forward the stream forpresentation of the content on a user interface such as a display screenand/or an audio speaker. The media client could take any of the formsdescribed above, among other possibilities.

The media source 14 could also take various forms, possibly depending onthe form of the media client 12. By way of example, if the media client12 is a television, the media source 14 could be a local set top box(e.g., cable or satellite receiver) or a streaming media server,configured to output a selected media stream for presentation by thetelevision, or the media source 14 could be a broadcast televisionstation or the like, among other possibilities. And if the media clientis a computer, the media source may comprise an Internet streaming mediaserver or other media provider. Alternatively, the media source 14 couldbe data storage integrated with and/or accessible locally by the mediaclient 12, with the media client receiving (e.g., retrieving) the mediastream from the data storage for playout. Other examples are possible aswell.

As further shown in FIG. 1, media client 12 could be in communicationover a network (e.g., the Internet or a private network) 16 with anetwork server 18, which may or may not be associated with and/orintegrated with media source 14.

In an example implementation, media client 12 and network server 18could engage in data communication to facilitate carrying out operationssuch as those noted above. For instance, aspects of a representativecomputing system could be implemented by the server 18, interoperatingwith the client 12 to establish synchronous lock, to apply thesynchronous lock to determine a client-time point when the client shouldperform a content revision, and to cause the client to perform thecontent revision at the determined client-time point. Further oralternatively, aspects of a representative computing system could beimplemented by the client 12, interoperating with the server 18 to carryout similar operations.

As noted above, when a media client such as client 12 is receiving amedia stream, it may be useful for the client to revise a portion of thestream's content, such as to replace or supplement a segment of thestream's content. Further, as noted above, it may be desirable for theclient to do so at a specific time point within the media stream. Forinstance, if the stream defines a sequence of frames, such as videoframes and/or audio frames, it may be desirable for the client toperform the revision starting at a specific frame of the sequence.

To facilitate such content revision, a computing system could firstidentify the media stream that is being received by the client. Forinstance, the computing system could first determine that the mediastream being received by the client is one that contains particularcontent that should be revised. Upon identifying the media stream, thecomputing system could then determine a specific point in the mediastream where the client should perform the content revision and couldcause the client to perform the content revision at that determinedpoint.

One way to identify the media stream being received by a client is tohave the client (or perhaps an adjunct device) generate digitalfingerprint data representing the media content being received by theclient, and to have server 18 compare that fingerprint data withreference fingerprint data representing media content of a known mediastream, with a fingerprint match indicating that the client is receivingthe known media stream.

In an example implementation, server 18 or another such entity couldestablish or otherwise have access to reference fingerprint datarepresenting the known media stream. For instance, the server couldestablish or be provisioned with a sequence of reference digitalfingerprints representing frames of the media stream. The server oranother entity could establish these reference fingerprints byprogrammatically analyzing frames of the known media stream, receivingas input a copy of the known media stream and applying any mediafingerprinting process or algorithm now known or later developed togenerate the reference fingerprints. These reference fingerprints couldbe generated at a relatively high level of granularity to facilitatefingerprint matching. For example, for a media stream comprising videoat 30 frames per second, the sequence could include a reference videofingerprint of every frame, thus likewise at a rate of 30 frames persecond, or at another reasonably high rate.

Further, in real time as the client 12 receives a media stream, theclient could generate query fingerprint data representing the mediacontent of the stream that it is receiving and could transmit the queryfingerprint data to the server 18 for analysis. The client couldestablish these query fingerprints using the same fingerprinting processor algorithm applied to establish the reference fingerprints, tofacilitate fingerprint comparisons. However, as noted above, due toresource limitations at the client (e.g., limitations on processingpower, memory, and network speed) or for other reasons, and given thatthis query-fingerprint generation could be an ongoing process for theclient, the client could be configured to generate these queryfingerprints at a relatively low rate. For instance, for a media streamcomprising video, the client could be configured to generate these queryfingerprints at a rate of only 2 frames per second, and the client mighttransmit generated query fingerprints to the server every 1.5 seconds orso. Other examples are possible as well.

As the server receives the query fingerprint data from the client, theserver could compare the query fingerprint data with the referencefingerprint data of the known media stream. And if the server therebydetermines with sufficient confidence that the query fingerprint datamatches the reference fingerprint data, the server could conclude thatthe media stream being received by the client is the known media stream(i.e., a copy of the known media stream).

The server could conduct this fingerprint comparison on a per-framebasis. Considering video content, for instance, the server could conductthe analysis on a per-video-frame basis, comparing a query fingerprintof a frame against a reference fingerprint of a frame. The comparisonprocess could take any of a variety of forms, depending on the structureof the fingerprints. For example, the comparison could involve computinga maximum deviation between the fingerprints and determining if themaximum deviation is within a predefined tolerance. Further, if thefingerprints are binary, this could be a Boolean determination or couldinvolve computing a Hamming distance (as a count of mismatches betweenrespective bit locations in the fingerprints). And if the fingerprintsare more complex values, such as decimal values or vectors (e.g., greyvalues per video frame region), this could involve determining adistance between the values or vectors. Numerous other examples arepossible as well.

In addition, the server could conduct the fingerprint comparison processwith a desired level of tolerance. For instance, the comparison could becoarse, with relatively high tolerance for differences between thefingerprints, such as with tolerance for a greater number of differencesand/or tolerance for a greater size of differences. By way of example,when comparing two 32-bit fingerprints with each other, a coarsecomparison may deem the fingerprints to match each other generally evenif two or another designated number of bits do not match. Or thecomparison could be more granular, with lower tolerance for suchdifferences, and/or with comparisons focused on specific fingerprintcomponents possibly representing specific portions of the underlyingmedia.

Further, the server could conduct the fingerprint comparison withrespect to one query fingerprint at a time. For instance, the servercould compare each sequentially provided query fingerprint with thereference fingerprints in an effort to find a reference fingerprint thatmatches the query fingerprint within a defined matching tolerance.

Alternatively, the server could conduct the comparison with respect to abundle of fingerprints at a time. For instance, the server could compareeach sequential bundle of four consecutive query fingerprints (on asliding window basis, or as discrete, sequential groups) with thereference fingerprints in an effort to find a bundle of four referencefingerprints that match the group of four query fingerprints within adefined matching tolerance. To facilitate this, if the queryfingerprints and reference fingerprints are generated at different framerates than each other, the server could configure its search for amatching bundle of reference fingerprints that are spaced apart fromeach other in time the same as the bundle of query fingerprints are. Forinstance, if the query fingerprints are generated at 2 frame per second(i.e., 500 milliseconds apart from each other) and the referencefingerprints are generated at 30 frames per second, then server couldsearch for a bundle of four reference fingerprints that are 500milliseconds apart from each other and that match a group of fourconsecutive query fingerprints, within a defined matching tolerance.

When the server has detected matching content with sufficientconfidence, the server will therefore have identified one or morematching points in the media stream. If the server performed thefingerprint comparison using one fingerprint at a time, then eachmatching point would be a frame that the matching reference fingerprintrepresents. Whereas, if the server performed the fingerprint comparisonusing a group of fingerprints at a time, then each matching point couldbe a frame that the first or another designated reference fingerprint inthe matching group represents. In any event, this matching couldestablish or establish or support a conclusion that the media streambeing received by the client is the known media stream.

Alternatively, the server could identify the media stream at issue inother ways. For instance, if the media stream contains watermark datacorresponding with an identity of the media stream, the client couldread or otherwise extract the watermark data and report the watermarkdata to the server. And the server could then use the reported watermarkdata as a basis to identify the media stream being received and/orplayed by the client.

Having identified the media stream being received by the client, theserver could then determine a point in the media stream where the clientshould perform a content revision. In a representative implementation,for instance, the server could have access to reference data thatindicates the point in the known media stream where a content revisionis to occur and that indicates what content revision is to occur at thatpoint. For example, the reference data may identify a particular frameof the known media stream where a content revision is to start. Thus,the server could refer to the reference data to determine that desiredcontent-revision point.

The server could then transmit to the client a directive that causes theclient to perform the content revision at the determined point in themedia stream. For instance, the server could transmit to the client overnetwork 16 a message that specifies the determined point in the mediastream and that provides the client with replacement or overlay media orotherwise directs the client as to what content revision the client isto perform at that point in the media stream. And the client could beconfigured to receive and interpret the directive and to responsivelyperform the content revision at the indicated point in the media stream.

In this process, the server may know the particular time point where theclient should perform the content revision in the media stream. Forinstance, the reference data accessible by the server could specify thistime point as a particular time measured in a timeline within the knownmedia stream, i.e., within “true time” as described above. As notedabove, for example, this timeline might start with an origin time (e.g.,T=0) at a start of the known media stream, and each frame of the knownmedia stream might occur at a particular time point on that timeline.Thus, the server may determine from reference data the true-time pointwhere the client should perform the content revision in the mediastream.

However, the client's time reference may be different. As noted above,the client's time reference may be “client time” defined according to aninternal clock in the client. Although client time and true time mayoptimally run at the same rate as each other, their origin points wouldlikely differ. Therefore, mere knowledge of a true-time point in theknown media stream would not itself provide context in client time. As aresult, at issue as noted above is how the server could characterize ina client-understandable manner the time point in the media stream wherethe client is to start performing the content revision.

One way to address this issue as suggested above is to have the serveruse client-provided timestamps of matching query fingerprints as a wayfor the server to determine the client's time reference for the mediastream, and then have the server use that client time-reference as abasis to characterize the time point in the media stream where theclient is to start the content revision.

To facilitate this, as the client generates its query fingerprints, theclient could timestamp each query fingerprint based on the client'sinternal clock. Thus, the query fingerprints that the client transmitsto the sever for analysis could be timestamped query fingerprints.Further, the server could be provisioned with or otherwise have accessto timestamped reference fingerprints of the known media stream frames.For instance, the server could be provisioned with reference dataspecifying for each frame of the known media stream (i) a referencefingerprint of the frame and (ii) an associated true-time timestampwithin the known media stream.

As the server receives the timestamped query fingerprints from theclient, the server could compare the query fingerprints with thereference fingerprints in an effort to detect matching points, i.e.,instances where a query fingerprint matches a reference fingerprint.When the server has detected one or more such matching points, theserver will know for each detected matching point both (i) the true-timetimestamp of the matching point and (ii) the client-time timestamp ofthe matching point.

As such, the server will have effectively established a synchronous lockbetween true time and client time. Namely, the server will haveeffectively established a mapping between (i) true-time timestampswithin the media stream and (ii) client time according to the client'sinternal clock, as the server will now know both where in the mediastream the matching frame was and what the client's time was at thattime point in the media stream. As discussed above, this synchronouslock may constitute a time offset, such that the server could apply thetime offset to a given true-time point in the media stream to determinethe corresponding client-time point in the media stream.

Having found this matching point in the media stream and havingestablished the synchronous lock, the server could compute theclient-time point when the client is to perform the content revision inthe media stream, i.e., where the client is to start the content in themedia stream. For example, given the reference-data indication of thetrue-time point when the content revision is to occur, the server couldapply the offset defined by the synchronous lock, to determine theassociated client-time point when the content revision is to occur. Orthe server could compute a duration within the known media stream fromthe true-time point of the matching point to the time of the frame wherethe content revision is to start and the server could then add thatcomputed duration on to the client-time point of the matching point, inorder to determine the associated client-time point when the contentrevision is to start. Other examples could be possible as well.

The server could then transmit a specification of this determinedclient-time point to the client, with a directive for the client toperform the content revision starting at this client-time point. And ifthe client is not already provisioned with information about the contentrevision to perform starting at that time, the server could alsotransmit that content-revision information to the client. For instance,the server could retrieve from data storage and transmit to the clientcertain replacement content for the client to substitute in place ofcontent in the media stream being received by the client. Or the servercould transmit to the client a pointer to such replacement content, suchas a network or local storage address from which the client could obtainthe replacement content.

The client could thus receive from the server the directive to start thecontent revision at the indicated client-time point. And if the clientis not already provisioned with data defining the revision to perform atthat client-time point, the client could receive that data, such as fromthe server or from an address specified by the server. By reference tothe client's internal clock, the client could then work to accuratelystart the content revision at the indicated client-time point. Forinstance, at the indicated client-time point, the client could beginsubstituting frames of replacement content for frames of the mediastream being received by the client, or overlaying supplemental contenton frames of the media stream being received by the client, among otherpossibilities.

As noted above, however, another issue in this process can arise if theclient generates its query fingerprints at a relatively low rate, suchas video fingerprints at 2 frames per second as discussed above.Providing the server with query fingerprints generated at such a lowrate could make it difficult for the server to confidently identify amatching point in the media stream and/or could result in low-confidencesynchronous lock.

One reason for this difficulty as noted above is that the media streammay include various sub-sequences of identical or nearly identicalframes, corresponding with portions of the media sequence where themedia content does not change at all over a period of time. In thatscenario, a query fingerprint might match multiple consecutive referencefingerprints within the defined matching tolerance. Further, a bundle ofquery fingerprints might match multiple bundles of referencefingerprints within the defined matching tolerance (e.g., with somequery fingerprints in the bundle matching corresponding referencefingerprints more or less, possibly due to fading or other variations).As a result, the server's detecting of a match and establishing ofsynchronous lock may be relatively low-confidence.

Per the present disclosure, as noted above, this issue can be at leastpartly resolved by taking into account multiple detected matching pointsand interpolating between those points to establish an improved mappingbetween true-time and client-time.

In a representative implementation, as the server receives low-ratequery fingerprints from the client and conducts a fingerprintcomparisons as discussed above, the server will detect multiple matchingpoints over time (for each of various query fingerprints or for each ofvarious bundles of query fingerprints), with each matching pointrepresenting a determined correlation between a true-time point and aclient-time point. As the server detects these matching points, theserver could then programmatically establish a scatter plot of thepoints in Cartesian coordinates, with true time along one axis andclient time along the other axis, and the server could perform linearregression to establish a best-fit line based on the points.

FIG. 2 illustrates this by way of example. In particular, FIG. 2illustrates a scatter plot with true time along the x axis and clienttime along the y axis. The scatter plot includes multiple representativematching points, and the figure depicts a best-fit line establishedbetween the matching points. This best-fit line effectively representsan improved mapping, virtual timing, or synchronous lock between truetime and client time. By performing linear regression over a sufficientquantity of matching points, the server could thus establish thisimproved mapping when faced with low-rate query fingerprints.

Note that the scatter-plot and best-fit analysis described here does notneed to be done geometrically or visually. The server could carry outthe process algorithmically, using any linear regression algorithm nowknown or later developed, based on the true-time, client-timecoordinates of the detected matching points.

Note also that the best-fit line established in this process maytypically have a slope of 1, assuming that the media stream rate (framesper second) at the client is the same as at the server. Given this, atissue could be where the best-fit line sits vertically or horizontallyin the plot and thus what the time offset is between true-time andclient-time. In addition, there could be scenarios where the mediastream rate at the client differs somewhat from the media stream rate atthe server, in which case the slope of the line may be different than 1.

The server could then use this improved mapping as a basis to predictwhat point in client-time will correlate with an upcoming point intrue-time, such as with a reference timestamp of a frame where contentrevision or associated action is to occur. And the server could transmitto the client a directive that specifies the determined client-time andthat directs the client to carry out the desired action when thatclient-time arrives.

As a non-limiting example of this process, consider a media stream thatdefines an ad pod that could appear within a television broadcast. Asnoted above, the ad pod would comprise a sequence of ads, such astelevision commercials, and the ad pod could appear at any given time ina linear media feed, such as a linear broadcast feed, being received bythe client.

In practice, the server could be provisioned with timestamped referencefingerprints representing and corresponding with the frames of the adpod, with the timestamps being points along a timeline defined withinthe ad pod (as true-time). Further, the server could be provisioned withan indication that a particular one of the ads in the ad pod should bereplaced, and with a timestamp of the frame where that ad replacementshould start.

On an ongoing basis, or in response to one or more other triggers, asthe client is receiving a media feed, the client could be generatinglow-rate timestamped query fingerprints of the broadcast feed, such asat 2 frames per second, and could be reporting those timestamped queryfingerprints to the server for analysis. Comparing each such queryfingerprint with the reference fingerprints representing the frames ofthe ad pod sequence, the server could thereby detect that the client isreceiving the known ad pod sequence.

Having determined that the client is receiving the ad pod sequence, theserver could then work to establish the improved mapping between truetime and client time so as to then determine a client-time when theclient is to start the ad replacement.

For instance, the server could compare bundles of the reported queryfingerprints with bundles of correspondingly spaced referencefingerprints and could thereby find multiple matching points over time(possibly starting with one or more matching points that the serveridentified as a basis to detect in the first place that the client isreceiving the ad pod sequence). As discussed above, each matching pointwould correlate a true-time point (timestamp of a representativematching reference fingerprint) with a client-time point (timestamp of arepresentative matching query fingerprint). Further, the server couldperform linear regression between these matching points to find abest-fit line, effectively representing the improved mapping betweentrue time and client time, in the form of an offset between true-timeand client-time.

Having established the improved mapping between true time and clienttime, the server could then apply the mapping as a basis to determinewhat client-time point will correspond with the true-time point at whichthe ad replacement is to start. In particular, given knowledge of thetrue-time point (the reference time) of the frame where the adreplacement is to start, the server could apply the offset defined bythe improved mapping, to translate that true-time point to a client-timepoint.

The server could then transmit to the client a directive to start the adreplacement at that determined client-time point and could transmit toclient the replacement ad (or a pointer to the replacement ad) if theclient is not already provisioned with that information. And the clientcould then accordingly perform the ad replacement at the indicatedclient-time point.

Note that the process described above could be varied or enhanced innumerous ways.

By way of example, to help account for the possibility that a segment ofthe media stream is static (not materially changing over time), theserver could be configured to disregard any bundle of query fingerprintsthat match each other (i.e., within a defined matching-tolerance). Inparticular, as the server receives query fingerprints from the clientand seeks to compare successive bundles of query fingerprints, theserver could evaluate each such bundle of query fingerprints todetermine if the query fingerprints of the bundle match each other. Ifthe server determines that all of the query fingerprints of the bundlematch each other, then the server could responsively exclude that bundleof query fingerprints from the matching-point analysis. For instance,the server could responsively forgo seeking to find a bundle ofreference fingerprints that match that bundle of query fingerprints.Whereas, if the server determines that the query fingerprints of thebundle do not all match each other, then the server could proceed withthe matching-point analysis as discussed above, in an effort to find amatching point that the server could include in its scatter-plot andbest-fit analysis.

In this example, at issue could be whether all of the query fingerprintsin the bundle match each other. Such a match of low-rate queryfingerprints could arise with respect to a sufficiently long staticsegment of the media stream. It may thus be useful to exclude thatbundle of query fingerprints from the matching-point analysis, to helpavoid factoring in a very low-confidence matching point.

On the other hand, consider a scenario where the bundle of four low-ratequery fingerprints straddles a change from one static segment toanother, different static segment. For instance, with video content, thefirst two query fingerprints of the bundle might represent frames beforea scene change occurs, and the last two query fingerprints of the bundlemight represent frames after the scene change occurs. In that situation,the four query fingerprints of the bundle would not all match eachother, and so the server would include the bundle in its matching-pointanalysis and may then detect a matching point to include in itsscatter-plot and best-fit analysis. With this example, though, it wouldbe uncertain whether the matching point will fall right on the best-fitline or not, as it would be unclear where in time the scene changeoccurred between the frames represented by the second and third queryfingerprints. But again, with a sufficient quantity of matching points,the best-fit analysis should still produce a useful, improved mapping asdescribed above.

Thus, as another example, the server could require that there be atleast a threshold minimum quantity of matching points within the mediastream at issue, as a condition for the server proceeding with theinterpolation. For instance, the server could keep a count of how manymatching points it has found in the media stream at issue (e.g., in anad pod). Once the server determines that the number of matching pointsin the media stream meets a predefined threshold, the server couldresponsively perform the scatter-plot and best-fit analysis describedabove in order to establish the improved time mapping based on theidentified matching points and could then use that time mapping asdescribed above. In practice, the threshold minimum quantity of matchingpoints could be three matching points or could be another number ofmatching points deemed by engineering design to be sufficient for thispurpose.

As still another example, after the server starts its scatter-plot andbest-fit analysis, the server could continue to find matching points andrefine its best-fit time mapping until the server reaches a stoppingpoint when the server would deem the best-fit mapping to be useable. Thestopping point could take various forms. For instance, the stoppingpoint could be when the server has factored in a predefined thresholdquantity of matching points, such as the threshold minimum quantitydiscussed above. Alternatively or additionally, the stopping point couldbe when the server has detected stability in the analysis, such as bydetecting that the best-fit line (or the associated time offset betweentrue-time and client-time) does not change more than a minimal thresholdamount (e.g., predefined by engineering design) for each additionalmatching point factored into the analysis. Once the server detects thestopping point, the server could then stop collecting matching pointsand could proceed to use the latest best-fit analysis to map thetrue-time point where action (e.g., content revision) is to occur to anassociated client-time point. And the server could accordingly directthe client to perform the action at the determined client-time point.

As yet another example, in addition to using the above process todetermine the client-time point when a content revision or the like isto occur, the server could also use the process to determine an advancedclient-time point when it would be useful for the client to startpreparation for the content revision. For instance, if the client mayneed to start obtaining replacement or overlay content, throughstreaming or download from an Internet URL or other network address orsource, and if it may take some time for the client to actually obtainor start receiving that replacement or overlay content, the server couldcompute an advanced client-time point that is sufficiently ahead of thecontent revision start time, to leave enough time for the client toobtain or start receiving the replacement or overlay content. The servercould then transmit to the client a directive that specifies theadvanced client-time point and that causes the client to start at thatclient-time point the process of obtaining the replacement or overlaycontent, and the client could proceed accordingly. The client could thusobtain or start receiving the replacement or overlay content by the timethe client will need to perform the content revision.

Still further, as yet another example, although the above descriptionprovides for the server carrying out various operations, some or all ofthe operations could alternatively be carried out by the client or byanother device. For instance, although the above discussion describedthe server working to establish improved mapping between true time andclient time, and the server working to determine the client-time pointwhen a content revision or other action is to occur, the client could beconfigured to carry out some of those operations itself.

In a representative variation, for example, once the server hasdetermined that the client is receiving the known media stream and hasfound multiple matching points that correlate true time with clienttime, or as the server finds those matching points, the server couldreport the matching points to the client, advising the client of thetrue-time and client-time coordinates of each matching point. And theclient could then use that provided data to perform the scatter-plot andbest-fit analysis discussed above, so as to establish the improvedmapping between true time and client time. Further, the server couldprovide the client with a directive that specifies the true-time pointat which the client is to engage in a particular action (e.g., contentrevision or preparation for content revision), and the client couldapply the established mapping as discussed above to translate thattrue-time point to a client-time point. The client could then work tocarry out the indicated action at the determined client-time point.

Further, the client itself could carry out the fingerprint matchingprocess as well. For instance, once the server has determined that theclient is receiving the known media stream, the server could provide theclient with a set of timestamped reference fingerprints for an upcomingportion of the media stream and could inform the client of the referencetimestamp for the frame within the media stream where the desired actionis to occur.

For instance, once the server has determined that the client isreceiving the known media stream, the client could carry out the processto establish synchronous lock between (i) true-time timestamps in themedia stream and (ii) time according to the client's internal clock, andto determine the client-time point when the content revision is tooccur. To facilitate this in an example implementation, in addition todirecting the client to switch to high-rate fingerprinting, the servercould provide the client with a set of true-time timestamped referencefingerprints for an upcoming portion of the media stream and couldinform the client of the true-time timestamp for the frame within themedia stream where the content revision is to start.

The client could thus receive those timestamped reference fingerprintsfrom the server. And the client could compare its low-rate queryfingerprints with the provided reference fingerprints in an effort tofind matching points. As or after the client finds multiple matchingpoints, the client could then likewise perform the scatter-plot andbest-fit analysis discussed above, to establish the improved mappingbetween true time and client time. Further, the client could apply thatestablished mapping in the manner discussed above to determine aclient-time point where an indicated content revision or other action isto occur. And the client could then perform the indicated action at thedetermined client-time point.

Note also that the above processes could apply with respect to real-timeanalysis of a linear broadcast feed or the like being received by aclient (e.g., for present playout and/or storage) and could apply aswell with respect to analysis of a media stream being retrieved by theclient from data storage or the like (e.g., for present playout ortransfer). Other implementations are possible as well.

FIG. 3 is a flow chart depicting a method that could be carried out by acomputing system in line with the discussion above. Although operationsof the method are depicted in this figure as being carried out in aparticular sequence, some of the operations could be carried out inparallel with others, and other modifications are possible as well.

As shown in FIG. 3, at block 30, the computing system receives queryfingerprints representing a media stream that is being received by aclient, the query fingerprints being timestamped with client timeaccording to a clock of the client. Further at block 32, likely inparallel with block 30, the computing system compares the received queryfingerprints with reference fingerprints representing a known mediastream, the reference fingerprints being timestamped with true timeaccording to a timeline within the known media stream. And at block 34,the computing system performs linear regression based on the detectedmatching points, the linear regression establishing a mapping betweentrue time and client time.

At block 36, the computing system then applies the established mappingas a basis to determine a client-time point at which the client shouldperform an action with respect to the media stream being received by theclient. And at block 38, the computing system causes the client toperform, at the determined client-time point, the action.

In line with the discussion above, the action that the computing systemcauses the client to perform at the determined client-time point couldcomprises a content revision. For instance, the media stream beingreceived by the client could comprise an ad pod defining a sequence ofads, and the content revision could comprise replacing an ad of thesequence with replacement media content. Alternatively, the action couldcomprise preparation for such a content revision, such as initiating aprocess to acquire new content to facilitate the content revision. Otheractions are possible as well.

Further, as discussed above, the act of applying the established mappingas a basis to determine a client-time point at which the client shouldperform an action with respect to the media stream being received by theclient could comprise using the established mapping as a basis to map apredefined true-time point at which the action should be performed withrespect to the known media stream to a corresponding client-time pointat which the client should perform the action with respect to the mediastream being received by the client. For instance, given the true-timepoint of a frame in the known media stream where a content revisionshould occur, applying the established mapping could involve translatingthat true-time point to a corresponding client-time point at which theclient should perform the content revision in the media stream beingreceived by the client.

In addition, as discussed above, such a computing system could beimplemented by a server that is in network communication with theclient. In that case, the act of causing the client to perform theaction at the determined client-time point could involve transmittingfrom the server to the client a directive to which the client willrespond by performing the action at the determined client-time point.

Still further, as discussed above, each matching point in this processcould be based on a comparison between (i) a respective bundle of thequery fingerprints and (ii) the reference fingerprints. And in thatcase, the method could additionally include omitting aninternally-matching bundle from the process. For instance, the methodcould involve detecting that the query fingerprints of a given bundle ofthe query fingerprints match each other and, responsive to the detectingthat the query fingerprints of the given bundle match each other,excluding the given bundle from use to establish a matching point forthe linear regression.

Yet further, in line with the discussion above, the method could includethe computing system using one or more of the detected matching pointsas a basis to determine that the media stream being received by theclient is the known media stream. For instance, the computing systemcould make this determination initially and then, in response to atleast making this determination, the computing system could perform thelinear regression, apply the mapping, and cause the client to carry outthe action at the determined client-time point. Phrased another way, inthat situation, the performing of linear regression, the applying of themapping, and the causing of the client to perform the action at thedetermined client-time point could be collectively responsive to atleast the determining that the media stream being received by the clientis the known media stream.

FIG. 4 is another flow chart depicting a method that could be carriedout in accordance with the present disclosure, by the client or one ormore other entities. Here again, although operations of the method aredepicted as being carried out in a particular sequence, some of theoperations could be carried out in parallel with others, and othermodifications are possible as well.

As shown in FIG. 4, at block 40, the method includes ascertaining aplurality of matching points between (i) query fingerprints representinga media stream being received by a client and (ii) referencefingerprints, with each identified matching point defining a respectivematch between (i) a query fingerprint that is timestamped with clienttime defined according to a clock of the client and (ii) a referencefingerprint that is timestamped with true time according to a timelinewithin a known media stream.

Further, at block 42, the method includes performing linear regressionbased on the timestamps of the ascertained plurality of matching points,the linear regression establishing a mapping between true time andclient time. At block 44, the method then includes using the establishedmapping as a basis to determine a client-time point at which the clientshould perform an action with respect to media stream being received bythe client. And at block 46, the method include performing, at thedetermined client-time point, the action with respect to the mediastream being received by the client.

In line with the discussion above, the act of ascertaining the pluralityof matching points could take various forms. For instance, it couldinclude the client receiving a set of data indicating the plurality ofmatching points. In an example implementation as discussed above, aserver could perform fingerprint comparisons to determine the matchingpoints and could send to the client information about the matchingpoints, including client-time timestamp and true-time timestamp of eachmatching point, to enable the client to perform the linear regression.

Alternatively, as discussed above, the act of ascertaining the pluralityof matching points could involve the client receiving the true-timetimestamped reference fingerprints, the client generating theclient-time timestamped query fingerprints, and the client comparing thegenerated query fingerprints with the received reference fingerprints inorder to ascertain the matching points based on the comparing. Further,as discussed above, each matching point could be based on a comparisonbetween (i) a respective bundle of the query fingerprints and (ii) thereference fingerprints. And in that case, the method could additionallyinvolve detecting that the query fingerprints of a given bundle matcheach other and, responsive to detecting that the query fingerprints ofthe given bundle match each other, excluding the given bundle from useto establish a matching point for the linear regression.

In addition, as discussed above, the action with respect to the mediastream being received by the client could take various forms. Forinstance, the action could comprise a content revision, such as adreplacement in an ad pod or the like. Or the action could comprisepreparation for such a content revision, such as initiating contentacquisition to facilitate the content revision. Other examples arepossible as well.

Further, in line with the discussion above, the act of using theestablished mapping as a basis to determine a client-time point at whichthe client should perform the action with respect to the media streambeing received by the client could involve using the established mappingas a basis to map a predefined true-time point at which the actionshould be performed with respect to the known media stream to acorresponding client-time point at which the client should perform theaction with respect to the media stream being received by the client.

FIG. 5 is next a block diagram of an example computing system operableto carry out various operations described above. As noted above, aspectsof such a computing system could be implemented by media client 12and/or server 18, among other possibilities. Therefore, the blockdiagram of FIG. 4 could represent components of one or more suchentities.

As shown in FIG. 5, the example computing system includes a networkcommunication interface 52, a processing unit 54, and non-transitorydata storage 56, all of which could be coupled or communicatively linkedtogether through a network, message bus, and/or one or more otherconnection mechanisms 58.

Network communication interface 52 enables the computing system totransmit and receive. The network communication interface 52 couldcomprise wired and/or wireless Ethernet interface along with associatedcircuitry and software, among other possibilities. The processing unit54 could then comprise one or more general purpose processors (e.g.,microprocessors) and/or one or more specialized processors (e.g.,application-specific integrated circuits or the like). And thenon-transitory data storage 46 could comprise one or more volatileand/or non-volatile storage components, such as magnetic, optical, orflash storage.

As further shown in FIG. 5, the non-transitory data storage 56 stores(e.g., holds, is encoded with, or otherwise embodies or contains)program instructions 60 that are executable by the processing unit tocause the computing system to carry out operations such as thosedescribed above.

By way of example, these operations could include identifying aplurality of matching points between (i) query fingerprints representinga media stream being received by a media client and (ii) referencefingerprints, wherein each identified matching point defines arespective match between (i) a query fingerprint that is timestampedwith client time defined according to a clock of the media client and(ii) a reference fingerprint that is timestamped with true time definedaccording to a timeline within a known media stream. Further, theoperations could include performing linear regression based on thetimestamps of the identified plurality of matching points, the linearregression establishing a mapping between true time and client time. Theoperations could then include using the established mapping as a basisto determine a client-time point at which the media client shouldperform a content revision in the media stream being received by themedia client. And the operations could include causing the media clientto perform, at the determined client-time point, the content revision.

Further, various features described above can be incorporated in thiscontext, and vice versa. For instance, using the mapping could involvemapping a predefined true-time point of the content revision to acorresponding client-time point. And the content revision could involvecontent replacement, such as replacing an ad in an ad pod, or couldinvolve overlaying content, among other possibilities.

In addition, if the computing system is disposed at and/or implementedby a server, the act of causing the media client to perform the contentrevision at the determined client-time point could involve outputtingfor transmission to the media client, and/or sending to the mediaclient, a directive that causes the media client to do so. Further, theoperations could include excluding from the analysis a bundle of queryfingerprints that match each other. And the operations couldadditionally include using one or more of the identified matching pointsas a basis to determine that the media stream being received by theclient is the known media stream, in which case the other operationcould be collectively carried out in response that determination.

Alternatively or additionally, the operations could include ascertaininga plurality of matching points between (i) query fingerprintsrepresenting a media stream being received by a client and (ii)reference fingerprints, wherein each identified matching point defines arespective match between (i) a query fingerprint that is timestampedwith client time defined according to a clock of the client and (ii) areference fingerprint that is timestamped with true time definedaccording to a timeline within a known media stream. Further, theoperations could include performing linear regression based on thetimestamps of the ascertained plurality of matching points, the linearregression establishing a mapping between true time and client time. Theoperations could then include using the established mapping as a basisto determine a client-time point at which the client should perform anaction with respect to media stream being received by the client. Andthe operations could include performing, at the determined client-timepoint, the action with respect to the media stream being received by theclient.

Here too, various features described above can be incorporated in thiscontext, and vice versa. For instance, if the computing system isdisposed at and/or implemented by the client, the act of ascertainingthe matching points could involve receiving data indicating some or allof the matching points and/or could involve (i) generating the queryfingerprints, (ii) receiving the reference fingerprints, and (iii)comparing the query fingerprints with the reference fingerprints toascertain the matching points. Further, the action with respect to themedia stream being received by the client could comprise a contentrevision such as an ad replacement or could involve preparation for acontent revision.

In addition, various operations described above could be defined byprogram instructions stored on a non-transitory computer readablemedium, the program instructions being executable by a processing unitto carry out the operations (e.g., to cause the operations to be carriedout). Such a non-transitory computer readable medium could then beprovided, integrated, or otherwise implemented at the client, server,and/or another entity. Further, such operations could take other forms.

Exemplary embodiments have been described above. Those skilled in theart will understand, however, that changes and modifications may be madeto these embodiments without departing from the true scope and spirit ofthe invention.

What is claimed is:
 1. A method comprising: ascertaining a plurality ofmatching points between (i) query fingerprints representing a mediastream being received by a client and (ii) reference fingerprints,wherein each identified matching point defines a respective matchbetween (i) a query fingerprint that is timestamped with client timedefined according to a clock of the client and (ii) a referencefingerprint that is timestamped with true time according to a timelinewithin a known media stream; performing linear regression based on thetimestamps of the ascertained plurality of matching points, the linearregression establishing a mapping between true time and client time;using the established mapping as a basis to determine a client-timepoint at which the client should perform an action with respect to mediastream being received by the client; and performing, at the determinedclient-time point, the action with respect to the media stream beingreceived by the client, wherein each matching point is based on acomparison between (i) a respective bundle of the query fingerprints and(ii) the reference fingerprints, wherein a given bundle of the queryfingerprints is consecutive query fingerprints, and wherein the methodfurther comprises detecting that the consecutive query fingerprints ofthe given bundle of the query fingerprints match each other and,responsive to the detecting that the consecutive query fingerprints ofthe given bundle match each other, excluding the given bundle from useto establish a matching point for the linear regression.
 2. The methodof claim 1, implemented by the client.
 3. The method of claim 2, whereinthe ascertaining of the plurality of matching points comprises receivingby the client a set of data indicating the plurality of matching points.4. The method of claim 2, wherein the ascertaining of the plurality ofmatching points comprises: generating by the client the queryfingerprints; receiving by the client the reference fingerprints; andcomparing by the client the generated query fingerprints with thereceived reference fingerprints, and ascertaining the plurality ofmatching points based on the comparing.
 5. The method of claim 1,wherein the action with respect to the media stream being received bythe client comprises a content revision.
 6. The method of claim 5,wherein the media stream being received by the client comprises an adpod defining a sequence of ads, and wherein the content revisioncomprises replacing an ad of the sequence with replacement mediacontent.
 7. The method of claim 1, wherein the action with respect tothe media stream being received by the client comprises preparation fora content revision.
 8. The method of claim 1, wherein using theestablished mapping as a basis to determine a client-time point at whichthe client should perform the action with respect to the media streambeing received by the client comprises using the established mapping asa basis to map a predefined true-time point at which the action shouldbe performed with respect to the known media stream to a correspondingclient-time point at which the client should perform the action withrespect to the media stream being received by the client.
 9. A mediaclient comprising: a processing unit; non-transitory data storage; andprogram instructions stored in the non-transitory data storage andexecutable by the processing unit to cause the media client to carry outoperations including: ascertaining a plurality of matching pointsbetween (i) query fingerprints representing a media stream beingreceived by the media client and (ii) reference fingerprints, whereineach identified matching point defines a respective match between (i) aquery fingerprint that is timestamped with client time defined accordingto a clock of the media client and (ii) a reference fingerprint that istimestamped with true time according to a timeline within a known mediastream, performing linear regression based on the timestamps of theascertained plurality of matching points, the linear regressionestablishing a mapping between true time and client time, using theestablished mapping as a basis to determine a client-time point at whichthe media client should perform an action with respect to media streambeing received by the media client, and performing, at the determinedclient-time point, the action with respect to the media stream beingreceived by the media client, wherein each matching point is based on acomparison between (i) a respective bundle of the query fingerprints and(ii) the reference fingerprints, wherein a given bundle of the queryfingerprints is consecutive query fingerprints, and wherein theoperations further include detecting that the consecutive queryfingerprints of the given bundle of the query fingerprints match eachother and, responsive to the detecting that the consecutive queryfingerprints of the given bundle match each other, excluding the givenbundle from use to establish a matching point for the linear regression.10. The media client of claim 9, wherein the ascertaining of theplurality of matching points comprises receiving a set of dataindicating the plurality of matching points.
 11. The media client ofclaim 9, wherein the ascertaining of the plurality of matching pointscomprises: generating the query fingerprints; receiving the referencefingerprints; and comparing the generated query fingerprints with thereceived reference fingerprints, and ascertaining the plurality ofmatching points based on the comparing.
 12. The media client of claim 9,wherein the action with respect to the media stream being received bythe media client comprises a content revision.
 13. The media client ofclaim 12, wherein the media stream being received by the media clientcomprises an ad pod defining a sequence of ads, and wherein the contentrevision comprises replacing an ad of the sequence with replacementmedia content.
 14. The media client of claim 9, wherein the action withrespect to the media stream being received by the media client comprisespreparation for a content revision.
 15. The media client of claim 9,wherein using the established mapping as a basis to determine aclient-time point at which the media client should perform the actionwith respect to the media stream being received by the media clientcomprises using the established mapping as a basis to map a predefinedtrue-time point at which the action should be performed with respect tothe known media stream to a corresponding client-time point at which themedia client should perform the action with respect to the media streambeing received by the media client.
 16. A non-transitory computerreadable medium storing instructions executable by a processing unit tocarry out operations comprising: ascertaining a plurality of matchingpoints between (i) query fingerprints representing a media stream beingreceived by a client and (ii) reference fingerprints, wherein eachidentified matching point defines a respective match between (i) a queryfingerprint that is timestamped with client time defined according to aclock of the client and (ii) a reference fingerprint that is timestampedwith true time according to a timeline within a known media stream,performing linear regression based on the timestamps of the ascertainedplurality of matching points, the linear regression establishing amapping between true time and client time, using the established mappingas a basis to determine a client-time point at which the client shouldperform an action with respect to media stream being received by theclient, and performing, at the determined client-time point, the actionwith respect to the media stream being received by the client, whereineach matching point is based on a comparison between (i) a respectivebundle of the query fingerprints and (ii) the reference fingerprints,wherein a given bundle of the query fingerprints is consecutive queryfingerprints, and wherein the operations further include detecting thatthe consecutive query fingerprints of the given bundle of the queryfingerprints match each other and, responsive to the detecting that theconsecutive query fingerprints of the given bundle match each other,excluding the given bundle from use to establish a matching point forthe linear regression.
 17. The non-transitory computer-readable mediumof claim 16, implemented at the client.
 18. The non-transitorycomputer-readable medium of claim 16, wherein the action with respect tothe media stream being received by the client comprises revising contentof the media stream.